Using Concurrent Complication Reporting to Evaluate Resident Critical Thinking and Enhance Adult Learning.

OBJECTIVE: Critical thinking and accurate case analysis is difficult to quantify even within the context of routine morbidity and mortality reporting. We designed and implemented a HIPAA-compliant adverse outcome reporting system that collects weekly resident assessments of clinical care across multiple domains (case summary, complications, error analysis, Clavien-Dindo Harm, cognitive bias, standard of care, and ACGME core competencies). We hypothesized that incorporation of this system into the residency program's core curriculum would allow for identification of areas of cognitive weakness or strength and provide a longitudinal evaluation of critical thinking development. DESIGN: A validated, password-protected electronic platform linked to our electronic medical record was used to collect cases weekly in which surgical adverse events occurred. General surgery residents critiqued 1932 cases over a 4-year period from 3 major medical centers within our system. These data were reviewed by teaching faculty, corrected for accuracy and graded utilizing the software's critique algorithm. Grades were emailed to the residents at the time of the review, collected prospectively, stratified, and analyzed by post-graduate year (PGY). Evaluation of the resident scores for each domain and the resultant composite scores allowed for comparison of critical thinking skills across post-graduate year (PGY) over time. SETTING: Data was collected from 3 independently ACGME-accredited surgery residency programs over 3 tertiary hospitals within our health system. PARTICIPANTS: General surgery residents in clinical PGY 1-5. RESULTS: Residents scored highest in properly identifying ACGME core competencies and determining Clavien-Dindo scores (p < 0.006) with no improvement in providing accurate and concise clinical summaries. However, residents improved in recording data sufficient to identify error (p < 0.00001). A positive linear trend in median scores for all remaining domains except for cognitive bias was demonstrated (p < 0.001). Senior residents scored significantly higher than junior residents in all domains. Scores > 90% were never achieved. CONCLUSIONS: The use of an electronic standardized critique algorithm in the evaluation and assessment of adverse surgical case outcomes enabled the measure of residents' critical thinking skills. Feedback in the form of teaching faculty-facilitated discussion and emailed grades enhanced adult learning with a steady improvement in performance over PGY. Although residents improved with PGY, the data suggest that further improvement in all categories is possible. Implementing this standardized critique algorithm across PGY allows for evaluation of areas of individual resident weakness vs. strength, progression over time, and comparisons to peers. These data suggest that routine complication reporting may be enhanced as a critical thinking assessment tool and that improvement in critical thinking can be quantified. Incorporation of this platform into M&M conference has the potential to augment executive function and professional identity development.

The Surgical Learning Curve: Does Robotic Technical Skill Explain Differences in Operative Performance?

Background: Prior studies on technical skills use small collections of videos for assessment. However, there is likely heterogeneity of performance among surgeons and likely improvement after training. If technical skill explains these differences, then it should vary among practicing surgeons and improve over time. Materials and Methods: Sleeve gastrectomy cases (n = 162) between July 2018 and January 2021 at one health system were included. Global evaluative assessment of robotic skills (GEARS) scores were assigned by crowdsourced evaluators. Videos were manually annotated. Analysis of variance was used to compare continuous variables between surgeons. Tamhane's post hoc test was used to define differences between surgeons with the eta-squared value for effect size. Linear regression was used for temporal changes. A P value <.05 was considered significant. Results: Variations in operative time discriminated between individuals (e.g., between 2 surgeons, means were 91 and 112 minutes, Tamhane's = 0.001). Overall, GEARS scores did not vary significantly (e.g., between those 2 surgeons, means were 20.32 and 20.6, Tamhane's = 0.151). Operative time and total GEARS score did not change over time (R2 = 0.0001-0.096). Subcomponent scores showed idiosyncratic temporal changes, although force sensitivity increased among all (R2 = 0.172-0.243). For a novice surgeon, phase-adjusted operative time (R2 = 0.24), but not overall GEARS scores (R2 = 0.04), improved over time. Conclusions: GEARS scores showed less variability and did not improve with time for a novice surgeon. Improved technical skill does not explain the learning curve of a novice surgeon or variation among surgeons. More work could define valid surrogate metrics for performance analysis.

Reducing surgical site infections after colectomy: bundle item compliance, process, and outlier identification.

BACKGROUND: The purpose of this study was to implement a checklist monitoring system and identify critical surgical checklist items associated with post-colectomy surgical site infections (SSI). The relationship between checklist compliance, infection rates, and identification of non-compliant surgeons was explored. MATERIALS AND METHODS: National Health Safety Network (NHSN) data were imported annually to establish baseline incidence of post-colectomy SSI from 2016 to 2019. A colectomy checklist was used to monitor compliance for 1694 random colectomies (1274 elective; 420 emergency). Reports were generated monthly to profile system, hospital, surgeon-specific infection, and checklist compliance rates. RESULTS: Checklist compliance improved in elective and emergent colectomies to > 90% for all items except oral antibiotic and mechanical bowel prep in elective cases. Annualized total SSI and organ space infection rates in elective cases decreased by 33% and 45%, respectively. Elective and emergency SSI's were reduced for Superficial Incisional Primary (SIP), Deep Incisional Primary (DIP), and Intra-Abdominal Abscess (IAB) by 66%, 60.4%, and 78.3%, respectively. Checklist compliance between low (< 3%) and high (> 3%) infection rate surgeons demonstrated significantly lower utilization of oral antibiotic prep (p < 0.03) and mechanical bowel prep (p < 0.02) in high infection rate surgeons. CONCLUSION: Surgeons compliant with colectomy checklists decreased elective and emergency colectomy infection rates. Ceiling compliance rates > 95% for bundle items are suggested to achieve optimal reductions in SSIs and efforts should be focused on surgeons with NHSN infection rates > 3%. Oral antibiotic prep and mechanical bowel prep compliance rates in elective colectomy appeared to differentiate high infection rate surgeons from low infection rate surgeons.

Cognitive bias and severity of harm following surgery: Plan for workflow debiasing strategy.

INTRODUCTION: This study analyzes the relationship between cognitive bias (CB) and harm severity as measured by Clavien-Dindo Scores (CD). METHODS: A prospectively collected series of 655 severity matched general surgical cases with complications were analyzed. Cases were evaluated for CB and assigned harm scores as defined by CD grade. Potentially mitigating "debiasing" strategies were identified for each bias attribution. RESULTS: Among cases with CB, 24% (55/232) were CD(I-II) and 76% (177/232) were CD(III-V). Odds ratio suggests that serious complications occur nearly 60% more frequently when CB is identified. The CBs identified with severe harm were Overconfidence, Commission, Anchoring, Confirmation, and Diagnosis Momentum. Preliminary data on debiasing strategies suggest diagnosis review, linear reasoning and Type II thinking may be relevant in over 85% of complications. CONCLUSION: The incidence of CB is increased in patients sustaining severe harm. Understanding the specific CBs identified and their mitigating debiasing strategies may improve outcomes.

Impact of Timing of Tocilizumab Use in Hospitalized Patients With SARS-CoV-2 Infection.

BACKGROUND: SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) continues to be a global challenge due to the lack of definitive treatment strategies. We sought to determine the efficacy of early administration of anti-interleukin 6 therapy in reducing hospital mortality and progression to mechanical ventilation. METHODS: This was a retrospective chart review of 11,512 patients infected with SARS-CoV-2 who were admitted to a New York health system from March to May 2020. Tocilizumab was administered to subjects at the nasal cannula level of oxygen support to maintain an oxygen saturation of >88%. The Charlson comorbidity index was used as an objective assessment of the burden of comorbidities to predict 10-year mortality. The primary outcome of interest was hospital mortality. Secondary outcomes were progression to mechanical ventilation; the prevalence of venous thromboembolism and renal failure; and the change in C-reactive protein, D-dimer, and ferritin levels after tocilizumab administration. Propensity score matching by using a 1:2 protocol was used to match the tocilizumab and non-tocilizumab groups to minimize selection bias. The groups were matched on baseline demographic characteristics, including age, sex, and body mass index; Charlson comorbidity index score; laboratory markers, including ferritin, D-dimer, lactate dehydrogenase, and C-reactive protein values; and the maximum oxygen requirement at the time of tocilizumab administration. Mortality outcomes were evaluated based on the level of oxygen requirement and the day of hospitalization at the time of tocilizumab administration. RESULTS: The overall hospital mortality was significantly reduced in the tocilizumab group when tocilizumab was administered at the nasal cannula level (10.4% vs 22.0%; P = .002). In subjects who received tocilizumab at the nasal cannula level, the progression to mechanical ventilation was reduced versus subjects who were initially on higher levels of oxygen support (6.3% vs 18.7%; P < .001). There was no improvement in mortality when tocilizumab was given at the time of requiring non-rebreather, high-flow nasal cannula, noninvasive ventilator, or invasive ventilator. CONCLUSIONS: Early use of anti-interleukin 6 therapy may be associated with improved hospital mortality and reduction in progression to more severe coronavirus disease 2019.

Cognitive Bias Impact on Management of Postoperative Complications, Medical Error, and Standard of Care.

BACKGROUND: Cognitive bias (CB) is increasingly recognized as an important source of medical error and up to 75% of errors in internal medicine are thought to be cognitive in origin (O'Sullivan ED, Schofield SJ. Cognitive bias in clinical medicine. J R Coll Physicans Edinb. 2018;48;225-232). However, primary data regarding the true incidence of bias is lacking. A prospective evaluation of CB in the management of surgical cases with complications has not been reported. This study reports the incidence and distribution of various types of CBs, and evaluates their impact on management errors and standard of care (SOC). METHODS: A prospectively collected series of 736 general surgical cases with complications from three university hospitals was analyzed. Surgical residents evaluated cases for 22 types of CBs (Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78:775-780). Supervising quality officers validated all quality assessments. Data were assessed for the incidence of CBs, error assessments (diagnostic, technical, judgment, system, communication, therapeutic, and professionalism), and SOC. RESULTS: CB was attributed in 32.7% (241/736) of all cases with complications. The most common CBs identified, both singly and in groups, were anchoring, confirmation, omission, commission, overconfidence, premature closure, hindsight, diagnosis momentum, outcome, and ascertainment bias. The attribution of CB was correlated to a statistically significant increase in the incidence of management errors by the surgical team and lower SOC assessments. CONCLUSIONS: CBs are identified in the management of cases with complications and are associated with an increase in management errors and a degradation in SOC. Insight into the types of CBs and their association with the type and severity of management errors may prove useful in improving quality care.

Core Competency Self-Assessment Enhances Critical Review of Complications and Entrustable Activities.

BACKGROUND: The Accreditation Council for Graduate Medical Education has defined six core competencies (CCs) that every successful physician should possess. However, the assessment of CC achievement among trainees is difficult. This project was designed to prospectively evaluate the impact of resident identification of CC as a component of morbidity review on error identification and standard of care (SOC) assessments. The platform was assessed for its reliability as a measure of resident critical analysis of complication causality across postgraduate year (PGY). MATERIALS AND METHODS: A total of 1945 general surgery cases with complications were assessed for error identification and SOC management between January 1, 2016, and December 31, 2018. CC identification was additionally assessed between January 1, 2019, and December 31, 2019, and included 708 general surgery cases. Data were evaluated for error assessments and overall SOC management. PGY4 and 5 residents were compared for number of cases and complications reviewed, severity, error causation, and CC relevance. RESULTS: Study groups were equivalent by Clavien-Dindo scores. Error identification significantly increased in all categories: diagnostic (P < 0.001), technical (P < 0.05), judgment (P < 0.001), system (P < 0.001), and communication (P < 0.001). Overall SOC assessments validated by a supervising surgical quality officer were unchanged. An increased exposure to cases with severe complications, error causation, and CC relevance was noted across PGY. CONCLUSIONS: The addition of CC assessment into morbidity review appears to improve the critical thinking of evaluating residents by increasing the identification of management errors. Used as an element of prospective self-assessment, teaching residents to identify CC principles in cases with complications may assist in learner progression toward clinical competence and critical thinking.